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The Generator Matrix 


11.3.3 The Generator Matrix 


Here, we introduce the generator matrix. The generator matrix, usually shown by G, gives us an alternative way of 
analyzing continuous-time Markov chains. Consider a continuous-time Markov chain X(t ). Assume .X(O) = i. The 
chain will jump to the next state at time Tj , where Tj ~ Exponential ( A j). In particular, for a very small 
8 > 0 , we can write 

P(Ti < 5) = 1 - e~ x ' s 

« 1 - (1 - A iS) 

= A ,6. 

Thus, in a short interval of length 8, the probability of leaving state i is approximately A iS. For this reason, A j is often 
called the transition rate out of state i. Formally, we can write 


A i = lim 

<5—>0 + 


P(X(S) + i|X(0) = i) 


(11.7) 


Since we go from state i to state j with probability Pij , we call the quantity gij = XiPij, the transition rate from state 
i to state j. Here, we introduce the generator matrix, G, whose (i, j) th element is Qij, when % j. We choose the 
diagonal elements (gu) of G such that the rows of G sum to 0. That is, we let 


9ii 


j¥=i 

" Xip^ 

" Pij 


j¥=i 


= ~X i. 

The last equality resulted as follows: If Aj = 0, then clearly 


A i y j Pij Aj 0. 


If Aj 0, thenpjj = 0 (no self-transitions), so 

^2 Pij = L 

It turns out the generator matrix is useful in analyzing continuous-time Markov chains. 


Example 11.20 

Explain why the following approximations hold: 

a -Pjj(8) ~ 1 + gjjS, for all j 6 S. 
b-Pkj(8) ~ Sgkjjork ^ j. 
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• Solution 

o Let 8 be small. Eq uation 11.7 can be written as 

Pjj (8) ~ 1 - A jS 

= 1 + gjjS. 

Also, we can approximate Pkj( 8) = P(X(8) = j|X(0) = k ) as follows. This probability is 
approximately equal to the probability that we have a single transition from state k to state j in the interval 
[0,5]. Note that the probability of more than one transition is negligible if 8 is small (refer to the Poisson 
process section). Thus, we can write 

p kj {8) = P{X{S) = j\X{0) = k) 

~ P(X(8) ^ k\X(0) = k)p kj 

~ X k 8p k j 
= 8g kj , for k ± j- 


We can state the above approximations more precisely as 

a. g ]3 = - lim^o P f S) , for all j E 5; 


b. g kj = lim 


<5—>0 + 


PkM 


, for k ± j. 


The Generator Matrix 


For a continuous-time Markov chain, we define the generator matrix G. The (i, j)th entry of the transition matrix is 
given by 


9ij 


A iPij 

if * 7 L j 


if % = j 


Example 11.21 

Consider the continuous Markov chain of Example 11.17 : A chain with two states S = {0, 1} and 
A C ) = Ai = A > 0. In that example, we found that the transition matrix for any t > 0 is given by 

2A t ' 

2 e 

i„—2A t 

2 C J 

a. Find the generator matrix G. 

b. Show that for any t > 0, we have 

P\t) = P{t)G = GP(t), 



where P'(t) is the derivative of P(t). 
• Solution 

o a. First, we have 
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#00 — — Ao 

= -A, 

#n = —Ai 

= -A. 


The transition matrix for the corresponding jump chain is given by 


p = 

Poo 

Pol 

— 

"0 

1 ' 


-Pio 

Pu. 


1 

0 


Therefore, we have 


#01 — AoPoi 

= A, 

#10 = AyPio 

= A. 


Thus, the generator matrix is given by 


G 


-A A 
A -A 


b. We have 


P'(t) 


'-Xe~ 2Xt Xe~ 2Xt ' 
\e~ 2Xt —Xe~ 2Xt 


where P' (t) is the derivative of P(t). We also have 


P(t)G 


GP(t) 


'lil „—2\t 

2 ' 2 e 

1 1 p —2\t " 

2 2 

r-A 

A ' 


' —\e~ 2Xt 

\e~ 2Xt ' 

1 1 p —2\t 

-2 2 

l_i_l p—2Xt 

2 ' 2 e 

[ A 

-A 


_ \e~ 2Xt 

— \e~ 2Xt _ 

r-A a i 


"l_i_l p—2\t 

2 ' 2 e 

A - -e~ 
2 2 

2 At " 


' —\e~ 2Xt 

Xe~ 2Xt ' 

i 

V 

1 

V 


1 1 p—2\t 

.2 2 C 

1 + i e - 

2 ' 2 

2\t 


_ \e~ 2Xt 

—Xe~ 2Xt _ 


We conclude 


P'(t) = P(t)G = GP(t). 


The equation P'(t) = P(t)G = GP(t ) (in the above example) is in fact true in general. To see the proof idea, 
we can argue as follows. Let 8 be small. By Example 11.20 . we have 

Pjj (8) « 1 + g n 8, 

Pkj(8) & 8g kj , for k^j. 

Using the Chapman-Kolmogorov equation, we can write 
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Thus, 


Pij(t + 8) — Pjk ( t)pkj (<5) 

k€S 

kfj 

~ Pij(t)( 1 9jj^) + Pik(t)3gkj 

kfj 

Pijit') "I - ^Pij^P) 9jj "I - ^ ^ ^ Pik(t) 9kj 

kfj 

= Pijit ) + <5 ^ ] Pik(t)gkj- 

keS 


Pij(t + 8) - pij(t) 
8 


^ ] Pik {tfgkj, 

k&S 


which is the (i, j)th element of P{t)G. The above argument can be made rigorous. 

Forward and Backward Equations 


The forward equations state that 

P’(t) = P(t)G, 

which is equivalent to 

Pij{t) = Y^Ptkitfgkj, for all i,j G S. 

keS 


The backward equations state that 


P'(t) = GP(t), 


which is equivalent to 


p'ifi) = f° r all i,j e S. 

keS 

One of the main uses of the generator matrix is finding the stationary distribution. So far, we have seen how to find the 
stationary distribution using the jump chain. The following result tells us how to find the stationary matrix using the 
generator matrix. 

Consider a continuous Markov chain X(t) with the state space S and the generator Matrix G • The probability 
distribution 7T on S is a stationary distribution for X(t) if and only if it satisfies 

ttG = 0. 

Proof: 

For simplicity, let's assume that S is finite, i.e., 7T = [7Tq , 7Ti , - • - , 7T r ], for some T <E N. If 7T is a stationary 
distribution, then 7T = nP(t). Differentiating both sides, we obtain 
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d r . .. 

° = 

= *P'(*) 

= nGP(t) (backward equations) 

Now, let t = 0 and remember that _P( 0) = /, the identity matrix. We obtain 

0 = ttGP(O) = ttG . 

Next, let 7T be a probability distribution on S that satisfies 7 tG = 0. Then, by backward equations, 

P'(t) = GP(t). 

Multiplying both sides by 7T, we obtain 

7T P'(t) = irGP{t) - 0. 

Note that 7T P'(t) is the derivative of 7rP(t). Thus, we conclude 7rP(t) does not depend on t. In particular, for any 
t > 0 , we have 

7T P(t) = 7rP(0) = 7T. 

Therefore, 7T is a stationary distribution. 


Example 11.22 

The generator matrix for the continuous Markov chain of Example 11.17 is given by 


G 


-A A 
A -A 


Find the stationary distribution for this chain by solving 7 tG = 0. 

• Solution 

o We obtain 


irG = [7r 0 ,7ri 


-A 

A 


A 

-A 


0 . 


which results in 


7T 0 = 7Ti. 


We also need 


TTO +7Tl = 1- 


Solving the above equations, we obtain 
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Transition Rate Diagram: 

A continuous-time Markov chain can be shown by its transition rate diagram. In this diagram, the values gij are shown 
on the edges. The values of gu's are not usually shown because they are implied by the other values, i.e., 

9ii = — 'y ^ 9ij • 

For example, Figure 11.24 shows the transition rate diagram for the following generator matrix 



Figure 11.24 - The transition rate diagram for the continuous-time Markov chain defined by Equation 11.8. 

<— previous 
next —> 
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